Papers with textual entailment
Unsupervised Question Answering for Fact-Checking (D19-66)
Copied to clipboard
| Challenge: | Recent Deep Learning (DL) models have achieved human-level accuracy on natural language tasks such as question-answering, natural language inference, and textual entailment. |
| Approach: | They propose an unsupervised question-answering based approach for a similar task, fact-checking. |
| Outcome: | The proposed approach achieves label accuracy of 80.2% on the development set and 80.25% on the test set. |
Exploring Factual Entailment with NLI: A News Media Study (2024.starsem-1)
Copied to clipboard
| Challenge: | Recent studies have focused on the relationship between factuality and Natural Language Inference (NLI). |
| Approach: | They propose a novel annotation scheme that models factual rather than textual entailment and use it to annotate a dataset of naturally occurring sentences from news articles. |
| Outcome: | The proposed annotation scheme can be used to model factual relationships on a dataset of naturally occurring sentences from news articles. |
Embedding WordNet Knowledge for Textual Entailment (C18-1)
Copied to clipboard
| Challenge: | Existing deep learning models for textual entailment do not require any feature engineering or linguistic analysis. |
| Approach: | They propose to embed WordNet-derived lexical entailment relations into specially-learned word vectors and incorporate them into a decomposable attention model for textual enlightment. |
| Outcome: | The proposed model significantly improves on the SICK and SNLI datasets. |
BelarusianGLUE: Towards a Natural Language Understanding Benchmark for Belarusian (2025.acl-long)
Copied to clipboard
| Challenge: | Recent advances in NLP, such as large language models, have had groundbreaking impact on the field. |
| Approach: | They propose a benchmark for Belarusian, an East Slavic language, with 15K instances in five tasks: sentiment analysis, linguistic acceptability, word in context, Winograd schema challenge, textual entailment. |
| Outcome: | The proposed model underperforms on sentiment analysis, linguistic acceptability, word in context, Winograd schema challenge and textual entailment, but is competitive for linguistic acceptance. |
Understanding tables with intermediate pre-training (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Textual entailment is well studied, but is less well studied for table enlargement . a new dataset of millions of examples is used to train the model . |
| Approach: | They adapt a table-based BERT model to recognize entailment from a dataset . they evaluate table pruning techniques as a pre-processing step to improve model efficiency . |
| Outcome: | The proposed model improves training and prediction efficiency at a moderate drop in accuracy. |
TextHacker: Learning based Hybrid Local Search Algorithm for Text Hard-label Adversarial Attack (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing textual adversarial attacks use gradient or prediction confidence to generate adversarials, making it hard to be deployed in real-world applications. |
| Approach: | They propose a textual adversarial attack that randomly perturbs lots of words to craft an adversarial example. |
| Outcome: | The proposed attack outperforms existing hard-label attacks in terms of attack performance and adversary quality. |
ALERT: Adapt Language Models to Reasoning Tasks (2023.acl-long)
Copied to clipboard
Ping Yu, Tianlu Wang, Olga Golovneva, Badr AlKhamissi, Siddharth Verma, Zhijing Jin, Gargi Ghosh, Mona Diab, Asli Celikyilmaz
| Challenge: | Large language models have shown increasing in-context learning capabilities with scaling up the model and data sizes. |
| Approach: | They propose a benchmark and suite of analyses to evaluate reasoning skills of large language models. |
| Outcome: | The proposed model compares pre-trained and fine-tuned models on tasks that require reasoning skills to solve. |
X-PARADE: Cross-Lingual Textual Entailment and Information Divergence across Paragraphs (2024.naacl-long)
Copied to clipboard
| Challenge: | X-parade is the first cross-lingual dataset of paragraph-level information divergences and entailments . ability to recognize differences in meaning underlies many NLP tasks . |
| Approach: | They propose a cross-lingual dataset of paragraph-level information divergences . they use a dictionary to identify new or inferred information in Wikipedia pages . |
| Outcome: | The proposed dataset shows that the proposed methods fail to handle inferable information . the dataset contains fine-grained span-level annotations for content in different languages . |
ParsiNLU: A Suite of Language Understanding Challenges for Persian (2021.tacl-1)
Copied to clipboard
Daniel Khashabi, Arman Cohan, Siamak Shakeri, Pedram Hosseini, Pouya Pezeshkpour, Malihe Alikhani, Moin Aminnaseri, Marzieh Bitaab, Faeze Brahman, Sarik Ghazarian, Mozhdeh Gheini, Arman Kabiri, Rabeeh Karimi Mahabagdi, Omid Memarrast, Ahmadreza Mosallanezhad, Erfan Noury, Shahab Raji, Mohammad Sadegh Rasooli, Sepideh Sadeghi, Erfan Sadeqi Azer, Niloofar Safi Samghabadi, Mahsa Shafaei, Saber Sheybani, Ali Tazarv, Yadollah Yaghoobzadeh
| Challenge: | Despite progress in natural language understanding, most progress is concentrated on resource-rich languages like English . despite high-quality benchmarks, there are few available NLU datasets for Persian language . |
| Approach: | They propose a benchmark for Persian language that includes a range of language understanding tasks . they present their results on monolingual and multilingual pre-trained language models . |
| Outcome: | The proposed benchmarks compare human performance with monolingual and multilingual models on Persian language with high quality evaluation datasets. |
Two-Step Classification using Recasted Data for Low Resource Settings (2020.aacl-main)
Copied to clipboard
Shagun Uppal, Vivek Gupta, Avinash Swaminathan, Haimin Zhang, Debanjan Mahata, Rakesh Gosangi, Rajiv Ratn Shah, Amanda Stent
| Challenge: | Existing studies on NLP models focus on high resource languages like English, but there are only two datasets for Hindi. |
| Approach: | They propose a novel two-step classification method which uses textual-entailment predictions for classification task. |
| Outcome: | The proposed method improves classification performance by using a joint-objective for classification and textual entailment. |
End-Task Oriented Textual Entailment via Deep Explorations of Inter-Sentence Interactions (P18-2)
Copied to clipboard
| Challenge: | Existing datasets for textual entailment (TE) have been used to study TE. |
| Approach: | They propose a deep explorations of inter-sentence interactions for textual entailment task that uses a convolution to make important words in P and H play a dominant role in learnt representations. |
| Outcome: | Experiments show that the pretrained DEISTE on SciTail gets 5% improvement over prior state of the art and that it generalizes well on RTE-5. |
Logic Against Bias: Textual Entailment Mitigates Stereotypical Sentence Reasoning (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent studies show that textual entailment learning reduces social biases in pretrained sentence encoders. |
| Approach: | They compare pretrained sentence encoders with textual entailment models that learn language logic for downstream language understanding tasks. |
| Outcome: | The proposed models outperform models with lower bias without debiasing processes on stereotype, profession, and emotion bias tests. |
LIME: Weakly-Supervised Text Classification without Seeds (2022.coling-1)
Copied to clipboard
| Challenge: | Existing approaches to weakly-supervised text classification use only label names as sources of supervision. |
| Approach: | They propose a framework for weakly-supervised text classification that replaces seed-word generation with entailment-based pseudo-classification. |
| Outcome: | The proposed framework outperforms baselines and state-of-the-art in 4 benchmarks. |
Sentiment-Stance-Specificity (SSS) Dataset: Identifying Support-based Entailment among Opinions. (L18-1)
Copied to clipboard
| Challenge: | Argument mining is a method for extracting argument components and structures from natural language texts. |
| Approach: | They propose to model arguments as a set of premises that either support each other or collectively support a conclusion. |
| Outcome: | The proposed rules give an overall accuracy of 0.83 for the three datasets. |
KLEJ: Comprehensive Benchmark for Polish Language Understanding (2020.acl-main)
Copied to clipboard
| Challenge: | Recent introduction of robust, general-purpose models for fine-tuning has enabled improvements in general natural language understanding (NLU) but such benchmarks are only available for a handful of languages. |
| Approach: | They propose a multi-task benchmark for the Polish language understanding with an online leaderboard . they also propose GLUE, a task for named entity recognition and sentiment analysis . |
| Outcome: | The proposed model performs best on three out of nine tasks in the Polish language . the proposed model is also used in an e-commerce domain to analyze the sentiments of users . |
Semantically Distributed Robust Optimization for Vision-and-Language Inference (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to integrate linguistic knowledge into training pipelines are under-explored. |
| Approach: | They propose a model-agnostic method that leverages linguistic transformations to infer a set of linguistic phenomena. |
| Outcome: | The proposed method improves on benchmark datasets with images and video and is generalizable to other V&L tasks. |
Optimizing Word Segmentation for Downstream Task (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to optimize tokenizations for downstream tasks are not suitable for traditional NLP. |
| Approach: | They propose a method to explore a tokenization appropriate for a downstream task . they train a model to assign a high probability to such appropriate tokenization based on the downstream task loss . |
| Outcome: | The proposed method improves sentiment analysis and textual entailment tasks . it is also integrated into state-of-the-art contextualized embeddings and reports a positive effect . |
ConEntail: An Entailment-based Framework for Universal Zero and Few Shot Classification with Supervised Contrastive Pretraining (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing models for text classification are not universally applicable and lack annotated data. |
| Approach: | They propose a framework for universal zero and few shot classification with supervised contrastive pretraining that can generalize to diverse classification tasks in both zero and many shot settings. |
| Outcome: | The proposed framework outperforms baseline models in zero and few shot settings. |
Medical Knowledge-enriched Textual Entailment Framework (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to achieving robust medical question answering systems lack a textual entailment framework that can capture the con-text beyond the sentence. |
| Approach: | They propose a medical knowledge-enriched textual entailment framework that can acquire a semantic and global representation of the input medical text with the help of a relevant domain-specific knowledge graph. |
| Outcome: | The proposed framework achieves 8.27% improvement over existing language models on MEDIQA-RQE dataset. |
Entangled Relations: Leveraging NLI and Meta-analysis to Enhance Biomedical Relation Extraction (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent research has explored the potential of leveraging natural language inference (NLI) techniques to enhance relation extraction (RE). |
| Approach: | They propose a method that verbalizes relation classes into class-indicative hypotheses to align a traditionally multi-class classification task to one of textual entailment. |
| Outcome: | The proposed method improves relation extraction performance on BioRED and ReTACRED. |
Assessing Out-of-Domain Language Model Performance from Few Examples (2023.eacl-main)
Copied to clipboard
| Challenge: | Pretrained language models exhibit impressive generalization capabilities, but behave unpredictably under certain domain shifts. |
| Approach: | They propose to incorporate attributions into a few-shot model predicting out-of-domain (OOD) performance task to find out if models agree with pathological heuristics that may indicate worse generalization capabilities. |
| Outcome: | The proposed model-based model-learning model can perform better on a few-shot example set, and incorporate feature attributions to improve it. |
Deep Contextualized Word Representations (N18-1)
Copied to clipboard
Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, Luke Zettlemoyer
| Challenge: | a new type of deep contextualized word representation is proposed for language understanding problems . word vectors are learned functions of the internal states of a deep bidirectional language model . |
| Approach: | They propose a new type of deep contextualized word representation that models complex features of word use and how they vary across linguistic contexts. |
| Outcome: | The proposed representations improve the state of the art across six challenging NLP problems. |
LSOIE: A Large-Scale Dataset for Supervised Open Information Extraction (2021.eacl-main)
Copied to clipboard
| Challenge: | Open Information Extraction (OIE) systems extract factual propositions into n-ary tuples . current datasets are limited in size and diversity . |
| Approach: | They propose to convert QA-SRL 2.0 dataset to large-scale OIE dataset LSOIE. |
| Outcome: | The proposed dataset is 20 times larger than the next largest human-annotated OIE dataset. |
Game-theoretic Vocabulary Selection via the Shapley Value and Banzhaf Index (2021.naacl-main)
Copied to clipboard
| Challenge: | Using the full vocabulary results in less explainable and memory intensive models. |
| Approach: | They propose a vocabulary selection method that views words as members of a team trying to maximize the model's performance. |
| Outcome: | The proposed method outperforms baseline models on multiple tasks and datasets. |
Logic-guided Semantic Representation Learning for Zero-Shot Relation Classification (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods to zero-shot relation classification can only identify seen relations . existing methods rely on descriptive information to improve understandability of relation types . |
| Approach: | They propose a logic-guided semantic representation learning model for zero-shot relation classification that builds connections between seen and unseen relations via implicit and explicit semantic representations with knowledge graph embeddings and logic rules. |
| Outcome: | The proposed model can generalize to unseen relation types and achieve promising improvements. |
Learning to Model and Ignore Dataset Bias with Mixed Capacity Ensembles (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work has shown that datasets contain incidental correlations created by idiosyncrasies in the data collection process. |
| Approach: | They propose a method that detects and ignores dataset-specific correlations by introducing a new method that makes them conditionally independent. |
| Outcome: | The proposed method detects and ignores these kinds of dataset-specific correlations, and does not require the bias to be known in advance. |
SSMix: Saliency-Based Span Mixup for Text Classification (2021.findings-acl)
Copied to clipboard
| Challenge: | SSMix synthesizes a sentence while preserving the locality of two original texts by span-based mixing and keeping more tokens related to the prediction relying on saliency information. |
| Approach: | They propose a new method where the operation is performed on input text rather than on hidden vectors like previous approaches. |
| Outcome: | The proposed method outperforms hidden-level mixup methods on a wide range of text classification benchmarks including textual entailment, sentiment classification, and questiontype classification. |
Modelling Instance-Level Annotator Reliability for Natural Language Labelling Tasks (N19-1)
Copied to clipboard
| Challenge: | Existing models that estimate annotators' reliability only consider binary labels and multi-class labels. |
| Approach: | They propose an unsupervised model which can handle binary and multi-class labels and integrate neural networks to model the dependency between latent variables and instances. |
| Outcome: | The proposed model can handle binary and multi-class labels and can estimate reliability of annotators across instances. |
TINA: Textual Inference with Negation Augmentation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing transformer-based models perform poorly on textual entailment when examples contain negations. |
| Approach: | They propose a new definition of textual entailment that captures negation and a principled technique for negated data augmentation that can be combined with unlikelihood loss function. |
| Outcome: | The proposed method significantly improves on textual entailment datasets with negations without sacrificing performance on datasets without negation. |
Cross-Lingual Ability of Multilingual Masked Language Models: A Study of Language Structure (2022.acl-long)
Copied to clipboard
| Challenge: | Multilingual pre-trained language models have shown impressive cross-lingual ability. |
| Approach: | They argue that cross-language ability comes from commonality between languages . they create an artificial language by modifying property in source language . |
| Outcome: | The proposed model can be implemented in multilingual and low-resource language scenarios without cross-lingual supervision or aligned data. |
Learning from Child-directed Speech in Two-language Scenarios: A French-English Case-Study (2026.findings-eacl)
Copied to clipboard
| Challenge: | a systematic study of compact language models with limited computational resources is challenging for many research contexts and real-world applications. |
| Approach: | They extend BabyBERTa to English-French scenarios under strictly sizematched data conditions. |
| Outcome: | The proposed model extends to English-French scenarios under sizematched data conditions . the results show context-dependent effects of multilingual training . |
Contextualizing Language Models for Norms Diverging from Social Majority (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies on transformer-based language models have shown that there seems to be a 'moral dimension' to LMs, as they show high accuracy in related downstream tasks such as moral reasoning and action classification. |
| Approach: | They propose a mechanism based on deontic logic to allow for a flexible adaptation of individual norms by de-biasing training data sets and a task-reduction to textual entailment. |
| Outcome: | The proposed mechanism de-biases training data sets and reduces tasks to textual entailment. |
Semantic Relatedness Based Re-ranker for Text Spotting (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to text spotting are limited by semantic similarity, but they can be useful for other tasks. |
| Approach: | They propose a neural approach to learn semantic relatedness from existing sentences. |
| Outcome: | The proposed approach outperforms existing approaches when applied to a text spotting task. |
Abstract Meaning Representation-Based Logic-Driven Data Augmentation for Logical Reasoning (2024.findings-acl)
Copied to clipboard
Qiming Bao, Alex Peng, Zhenyun Deng, Wanjun Zhong, Gael Gendron, Timothy Pistotti, Neset Tan, Nathan Young, Yang Chen, Yonghua Zhu, Paul Denny, Michael Witbrock, Jiamou Liu
| Challenge: | Empirical evidence shows that our proposed method improves performance across seven downstream tasks. |
| Approach: | They propose a logic-driven data augmentation approach that converts text into AMR graphs and converts them back into text to create augmented data. |
| Outcome: | The proposed method leads on the ReClor leaderboard and improves on seven downstream tasks. |
ESCRITO - An NLP-Enhanced Educational Scoring Toolkit (L18-1)
Copied to clipboard
| Challenge: | Existing implementations are very specific to specific use cases and datasets. |
| Approach: | ESCRITO is a toolkit for scoring student writings using NLP techniques . authors propose teachers and NLP researchers to use APIs for scoring pipelines . |
| Outcome: | ESCRITO is a toolkit for scoring student writings using NLP techniques . it addresses two main user groups: teachers and NLP researchers . |
Is “My Favorite New Movie” My Favorite Movie? Probing the Understanding of Recursive Noun Phrases (2022.naacl-main)
Copied to clipboard
| Challenge: | Recursive noun phrases have interesting semantic properties, yet it is unknown whether language models have such knowledge. |
| Approach: | They propose a dataset of three textual inference tasks targeting recursive noun phrases . they show that such knowledge is learnable with appropriate data . |
| Outcome: | The proposed model achieves strong zero-shot performance on an extrinsic Harm Detection task. |
Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Recent models for unsupervised representation learning of text have put little focus on discourse-level representations. |
| Approach: | They propose an inter-sentence objective for pretraining language models that models discourse coherence and the distance between sentences. |
| Outcome: | The proposed model outperforms the BERT-Large model on the discourse representation benchmark DiscoEval and yields gains of 2%-6% absolute even for tasks that do not explicitly evaluate discourse. |
GeoSQA: A Benchmark for Scenario-based Question Answering in the Geography Domain at High School Level (D19-1)
Copied to clipboard
| Challenge: | SQA is an emerging application of NLP in the medical, geography, and legal domains. |
| Approach: | They propose a dataset of 1,981 scenarios and 4,110 multiple-choice questions in geography domain at high school level. |
| Outcome: | The proposed dataset consists of 1,981 scenarios and 4,110 multiple-choice questions in the geography domain at high school level. |
Baselines and Test Data for Cross-Lingual Inference (L18-1)
Copied to clipboard
| Challenge: | Recent research on textual entailment is limited to English, but it is expanding to other languages. |
| Approach: | They propose to extend the research in SNLI-style natural language inference toward multilingual evaluation by using cross-lingual word embeddings and machine translation. |
| Outcome: | The proposed system scores an average accuracy of just over 75%, but it is not perfect. |
Universal Natural Language Processing with Limited Annotations: Try Few-shot Textual Entailment as a Start (2020.emnlp-main)
Copied to clipboard
| Challenge: | a current approach to solving NLP problems is to build a problem-specific dataset . current approaches do not allow for transforming tasks into textual entailment . |
| Approach: | They propose a pretrained textual entailment system that can generalize across domains . they argue that when is it worth transforming an NLP task into textual detailment? |
| Outcome: | The proposed model can generalize across domains with few examples, the authors argue . they show that it can be used for several downstream NLP tasks with limited annotations . |
Cross-Policy Compliance Detection via Question Answering (2021.emnlp-main)
Copied to clipboard
| Challenge: | Policy compliance detection is the task of ensuring that a scenario conforms to a policy. |
| Approach: | They propose to decompose policy compliance detection into question answering . they propose to use an existing dataset to augment expert annotations . |
| Outcome: | The proposed approach improves accuracy in cross-policy setups, especially when policies are unseen in training. |
OpineSum: Entailment-based self-training for abstractive opinion summarization (2023.findings-acl)
Copied to clipboard
| Challenge: | Abstractive summarization is promising for fluently comparing opinions from a set of reviews about a place or product. |
| Approach: | They propose a novel method that automatically leverages common opinions across reviews to create powerful abstractive models. |
| Outcome: | The proposed method outperforms strong peer systems in both settings. |
Decomposing and Comparing Meaning Relations: Paraphrasing, Textual Entailment, Contradiction, and Specificity (2020.lrec-1)
Copied to clipboard
| Challenge: | SHARel is a new typology for decomposing and comparing multiple meaning relations . it consists of 26 linguistic and 8 reason-based categories and can be applied to all relations with a high inter-annotator agreement. |
| Approach: | They propose a new typology that consists of 26 linguistic and 8 reason-based categories and propose SHARel for decomposing and comparing multiple meaning relations. |
| Outcome: | The proposed method can be applied to all relations with high inter-annotator agreement. |
ARMAN: Pre-training with Semantically Selecting and Reordering of Sentences for Persian Abstractive Summarization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Abstractive summarization is one of the areas influenced by pre-trained language models. |
| Approach: | They propose a Transformer-based encoder-decoder model pre-trained with three novel objectives to address this issue. |
| Outcome: | The proposed model outperforms previous models on six Persian summarization tasks . it also outperformed previous models in textual entailment, question paraphrasing, and question answering . |
Japanese Realistic Textual Entailment Corpus (2020.lrec-1)
Copied to clipboard
| Challenge: | a corpus of 48,000 realistic examples is the largest among publicly available Japanese TE corpora . a textual entailment corpus is used to train natural language understanding . authors: to be truly helpful, machines must understand the meaning of texts. |
| Approach: | They perform textual entailment corpus construction with 48,000 realistic examples . they use two sentences that are spontaneous or almost equivalent . |
| Outcome: | The resulting corpus consists of 48,000 realistic Japanese examples . it is the largest among publicly available Japanese TE corpora . |
Explaining Mixtures of Sources in News Articles (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study shows that language models are essential for long-form article generation. |
| Approach: | They propose a generative process where a source-selection schema is first selected by a journalist, and then sources are chosen based on categories in that schema. |
| Outcome: | The proposed model can predict the most suitable schema given just the headline with reasonable accuracy. |
EmbodiedBERT: Cognitively Informed Metaphor Detection Incorporating Sensorimotor Information (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for metaphor detection rely on heuristics such as Metaphor Identification Procedure (MIP) and Selection Preference Violation (SPV). |
| Approach: | They propose a cognitively motivated module that leverages the cognitive information of embodiment that can be derived from word embeddings and explicitly models the process of sensorimotor change that has been demonstrated as essential for metaphor processing. |
| Outcome: | The proposed module can improve metaphor detection compared with the heuristic MIP that has been applied previously. |
On the Use of Silver Standard Data for Zero-shot Classification Tasks in Information Extraction (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing zero-shot methods for information extraction use large amounts of gold standard data. |
| Approach: | They propose a framework to utilize silver data to enhance zero-shot classification methods . they propose to use off-the-shelf models of other NLP tasks to perform inference on test data . |
| Outcome: | The proposed framework outperforms baseline methods on TACRED and Wiki80 datasets by 5% and 6% on the zero-shot relation classification task and by 3% 7 % on Smile (Korean and Polish) |
Contrastive Policy Gradient: Aligning LLMs on sequence-level scores in a supervised-friendly fashion (2024.emnlp-main)
Copied to clipboard
Yannis Flet-Berliac, Nathan Grinsztajn, Florian Strub, Eugene Choi, Bill Wu, Chris Cremer, Arash Ahmadian, Yash Chandak, Mohammad Azar, Olivier Pietquin, Matthieu Geist
| Challenge: | Reinforcement Learning (RL) is a method used to fine tune Large Language Models (LLMs) using a reward model trained from preference data to better align with human judgment. |
| Approach: | They propose a Reinforcement Learning (RL) algorithm that can estimate the optimal policy even from off-policy data. |
| Outcome: | The proposed algorithm can estimate the optimal policy even from off-policy data. |